1. Field
This disclosure relates to motor learning and rehabilitation using kinesthetic feedback.
2. Description of the Related Art
Roughly 8% of Americans have some motor skill inability. Victims of neurological trauma such as stroke form an especially interesting segment of the disabled population, because after stroke, the victims still possess adequate muscle mass, but no way to control it. The longer muscle retraining requires, the more injury can result to the body through accidents or body misuse.
On the other end of the spectrum, sports players and artists such as dancers depend on accurate motion to perform. Those that have the greatest control over their motor system perform best, and learning this control can take many years.
Typically both of these segments of the population improve their skills with a teacher—a professional who helps them improve their motor system skills. The novice/patient has three primary communication channels by which to learn new skills: auditory, usually providing high level information, or current state of performance; visual, by watching the teacher, and themselves as they attempt to perform; and tactile, through both the touch of a teacher in helping along certain motions, and in kinesthetic knowledge about one's own position/movement. A teacher cannot, however, provide real-time feedback for a novice in an efficient manner—humans have too many joints functioning in parallel, a teacher gets ‘in the way’ while touching a student, and a teacher uses subjective evaluation of performance, some aspects of which can be better examined by a computer vision system for accuracy.
Motor learning has been a subject of active research for over fifty years, and yet no deep understanding of mechanisms and methods has been found. Historically the study of motor skills learning came after World War II, when devices were developed to help Air Force pilots gain more information while flying, without requiring visual attention such as tilt readings. As early as the late forties it was known that feedback played an important role in motor learning. Below details describe the nature of motor feedback, and its importance in learning.
One point of note in agreement in the study of motor skill development is that feedback is crucial to levels of performance. Feedback provides three necessary components in learning: reward, information, and motivation. Even the knowledge of the sign of the errors in a ranging task was shown to improve a gunnery trainer's performance.
The time at which feedback is given is also extremely influential in human performance. Timely or near real-time feedback greatly enhances behavior and motor skill learning. It has been found that a student's ability to use feedback is seriously disrupted or made impossible if the feedback lags performance by 1.0 seconds or more.
The primary touch organ, skin, is the largest organ of the human body. The skin is very sensitive to periodic applied pressure, or vibration. Within this document, the application of period pressure to the skin will be referred to as vibrotactile stimulation. The sensitivity of the skin varies with the frequency of the applied pressure. Skin experiences its highest response to inputs of 250 Hz, and falls off at higher frequencies. Furthermore, our frequency sensitivity is sensitive to contactor size (the size of the element contacting the skin). Smaller contactors possess a flatter curve of sensitivity across frequencies, indicating that when small contactors are used the threshold response is independent of frequency. At higher frequencies (e.g., 80-230 Hz), sensitivity increases directly with contactor size.
We respond to frequencies differently in different ranges, especially differentiating between below and above 100 Hz. Furthermore, frequency content (harmonics) plays a role in tactile identification. Our ability to discriminate between frequencies is reasonable at lower frequencies but deteriorates rapidly as frequency is increased. The skin is rather poor at frequency discrimination. Pulses produce better discrimination than do sine waves.
The skin's ability to detect short signals depends on contactor size. The ability to detect gaps on skin impulses exists until roughly 10 ms, but is as low as 5 ms for highly damped mechanical pulses. Bursts of sinusoids are significantly easier to detect than bursts of noise. Sinusoids are felt as ‘smooth’ and the gap is perceived as a small click, whereas noise feels ‘rough’ and the gap is perceived as a modulus of stimulus amplitude.
When stimulus elements for sequences is increased to five or six, stimulus onset intervals needed for correct identification of the temporal sequence may be nearly 500 ms. This results in a ‘too slow’ perception for real time speech, for example. However, to simply discriminate between two temporal sequences with no requirement to identify temporal order, increasing the number of stimulus elements has little effect on performance and discrimination thresholds are generally below 100 ms.
One amazing aspect possessed by our somatosensory system is known as sensory saltation. It is best described with an example: We place three tactile transducers on the skin, one at the wrist, one ten cm up the arm, and one ten cm further. We apply five brief pulses to the wrist, then without any break in the regularity of the pulses, five more on the second transducer, and then five at the final transducer. The taps will seem to be distributed, contrary to our default bias, and seem to be spaced uniformly, more or less, from the region of the first contactor to that of the third.
Several conditions are necessarily met to create this saltatory illusion. Although even two pulses per location is adequate, the effect is most pronounced with four-six pulses per location. Any irregularity of the pulse sequence timing disturbs what has become called the ‘cutaneous rabbit.’Contactors can be placed as close as 2 cm apart, and as far as 35 cm apart, while still causing the hopping effect.
Although regularity in timing is very important, the timing between taps is not highly critical. A pronounced effect occurs over a wide range of interstimulus interval values (ISIs). We begin to notice the effect with an ISI of 200 msec, and it settles into an evenness at 100 msec. Upon reaching 50 msec ISI, the hopping effect is optimal in regularity and vividness. It has been found that with further shortening of the ISI, the perceived number of taps becomes illusory.
The saltation effect works equally well with electro-tactile stimulation as with vibrotactile pulses. When receiving these signals, often there is the impression that the taps extend beyond the terminal contactor. This effect is related to ‘synthetic movement’ and the ‘phi phenomenon’ present in the visual sensory system.
A great deal of work has been done in the last decade studying the benefits of augmented feedback, primarily given visually through a Virtual Reality (VR) environment. A key factor of motor learning is that motor repetition is not enough to “induce cortical correlates of motor learning.” The practice done by the subject must be linked to incremental success at some task or goal. Trial and error practice with feedback about performance success accomplishes this, with feedback gained through the senses.
Augmented feedback can enhance the cortical changes associated with motor learning. Virtual reality is one methodology by which we can add augmented feedback, but none of the gains have been shown to be peculiar to VR. With augmented feedback, we receive both proprioceptive (one's sense of body position) and exteroceptive (one's sense of stimuli outside of the body) feedback associated with the execution of a task, which induces profound cortical and subcortical changes at the cellular and synaptic level. Visual recognition of a teacher performing a task correctly stimulates mirror neurons for learning.
Typically, the augmented feedback given is a visual display of the subject's motion, as well as a visual display of the ‘correct’ motion, as performed by a coach or teacher. Both motions are tracked in real-time so the user at all times can see how their motion differs from the desired motion. In stroke motor rehabilitation experiments, not only did motions learned in VR translate into the real world, but they also generalized to motor learning in untrained spatial locations.
Learning to perform a task consists of two primary parts:                Finding the set of constraints that any movement must satisfy for success        Selecting a subset of movements easiest to produce and control to perform reliably        
These movements are known as task related invariants.
One possible way to teach task constraints is to provide reference movements that satisfy the constraints. Therefore, one role of augmented feedback, it has been suggested, might be to emphasize the differences between the subject's movements and the reference movement. There is psychophysical evidence that humans derive specifications of movement by tracking end-effector trajectories (of the limb, usually). By explicitly showing this trajectory through a VR display, learning may be enhanced, especially in the initial phase.
Tactors (tactile actuators) are transducers that were originally developed for sensory substitution, primarily for the deaf-blind community. By applying force to the skin, we can transmit coded information. The initial projects that accomplished this were such as the Teletactor (developed in 1931 by Robert Harvey Gault), an array of 32 actuators presenting sound, the Optacon (developed in the 1960s by Dr. James Bliss), a 6×24 array of actuators responding to light input (to translate written text into tactile stimulation), and the Videotact (produced in 1996 by the Unitech Company), which possess 768 electro-tactile actuators to present video. These devices ‘substituted’ a tactile channel for the more typical auditory and visual channels one would use to process such information.
The historical development of tactile interfaces always focused on this channel substitution, relegating visual or auditory information to the somatosensory channel.